Proactive path quality reporting in packet transmission

ABSTRACT

An originated packet is to be forward-propagated along a path to a destination device, with the path including one or more intermediary node devices. A path quality indication representing a reported error having been detected by at least one of the intermediary node devices residing along the path, is incorporated in the forward-propagating packet. The destination device incorporates the path quality indication in an acknowledgement packet directed to the originator of the packet.

TECHNICAL FIELD

Aspects of the disclosure relate generally to information processing andcommunications and, more particularly, to network routing and pathselection.

BACKGROUND

In high-speed networking, data throughput and low latency are keyobjectives for system designers. Oftentimes, networking equipment, suchas switches, routers, and other node devices, are pushed to theirphysical limit to attain networking performance goals. As a consequence,certain links may become prone to errors. Errors may be managed usingdata-encoding techniques such as forward error correction, for example,where a receiving node or intermediate-hop node detects and, sometimeseven corrects, errors in received data. Errors that cannot be correctedgenerally result in a negative acknowledgement (NACK) reported to thesending node, which responds by retransmitting the packet.Retransmissions tend to reduce data throughput, increase latency, andcontribute to congestion at affected portions of the network.

Conventionally, networks manage congestion by dropping packets, orotherwise calling for, or creating conditions in which, packets arere-routed using other available paths that are less congested. Onechallenge with this type of approach is that, by the time a link isdeemed congested, there has already been some period of time duringwhich the data-throughput performance of the network suffered a reducedlevel of data-throughput performance.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numeralsmay describe similar components in different views. Like numerals havingdifferent letter suffixes may represent different instances of similarcomponents. The drawings illustrate generally, by way of example, butnot by way of limitation, various embodiments discussed in the presentdocument.

FIG. 1A is a block diagram illustrating a portion of a computer networkin which aspects of the disclosure may be employed, and portions ofwhich may constitute embodiments.

FIG. 1B is a diagram illustrating a switched-fabric network topology inwhich aspects of the disclosure may be employed.

FIG. 2 is a block diagram illustrating a host machine with a networkinterface device according to embodiments.

FIG. 3 is a block diagram illustrating certain components of source nodethat obtains, and responds to, path quality indicia according to anexample embodiment.

FIG. 4 is a block diagram illustrating certain components of anintermediary node along a path of a packet according to an exampleembodiment.

FIG. 5 is a block diagram illustrating certain components of adestination device according to an example embodiment.

FIG. 6 is an information flow diagram illustrating a method of operationof source device, a destination device, and intermediate devices thatconstitute at least a portion of a path between the source anddestination devices, according to an example embodiment.

DETAILED DESCRIPTION

FIG. 1A is a block diagram illustrating a portion of a general computernetwork 100 in which aspects of the disclosure may be employed, andportions of which may constitute embodiments as will be described ingreater detail below. Computer network 100 includes various hostmachines 102, 106, 116, and 122, as depicted, which are communicativelycoupled with one another via one or more transmission media. Hostmachines 102, 106, 116, and 122 may be general-purpose computer systems,specialized computer system components, such as processing units, memoryunits, peripherals, storage devices, and the like. The term“transmission medium” shall be taken to include any tangible orintangible medium that is capable of storing, encoding, or carryinginstructions for execution by the machine, and includes digital oranalog communications signals or other intangible medium to facilitatecommunication of such software. Examples of transmission media includecopper wire, fiber optic cabling, wireless transmission channels, andthe like.

As a generally richly-featured example, a host machine's architecture(in the example case where a host machine is a general-purpose computersystem) is described below with reference to FIG. 2. It will beunderstood that different subsets of the system described with referenceto FIG. 2, or a system with an altogether different architecture, may beused to realize a host machine according to one or more embodimentsdescribed herein.

For the sake of simplicity in the networked context illustrated in FIG.1A, each host machine 102, 106, 116, and 122 includes a networkinterface device (NID) 104, 108, 118, and 124, respectively. Network 100is arranged such that host machines 102 and 106 are organized in subnetA with switch 110, and host machine 116 is part of subnet B havingswitch 120. Switches 110 and 120 are each coupled to local networkrouter 112, which in turn is coupled to wide-area network router 114,which provides connectivity to wide-area network 115, which may be theInternet according to one embodiment, or a proprietary wide-area networkaccording to another embodiment. In a related embodiment, switches 110,120 may each be coupled to wide-area network router 114 directly. Asillustrated, switch 120 is coupled to wide-area network router 114.Remote host 122 is coupled to wide-area network 115 via router 126 asshown.

In one example, host machine 102, which is also referred to herein aslocal host machine 102, is programmed or otherwise commanded underprogram control, to transact with remote host 122.

It will be understood that communications may be delivered acrossnetwork 100 as transactions, or segments, which may be handled aspackets, which in turn may be handled as frames of bits. A hierarchicalhandling of the transmission may be implemented, with differentprocesses carried out by the various nodes along the transmissionpath(s) on corresponding portions of the data of each transaction.

A variety of network topologies in which aspects of the embodiments maybe embodied are contemplated including, for instance, peer-to-peertopologies, mesh topologies, switched-fabric topologies, clustertopologies, and the like. In various example embodiments, an Infinibandor Omni-Path switched-fabric network is employed.

FIG. 1B is a diagram illustrating an example portion of aswitched-fabric network 150 in which certain embodiments may beimplemented. Host machine 152 communicates with host machine 156 overfabrics 160 and 162 as a first path 170, and over fabrics 164 and 166 asa second path 172. Each fabric 160-166 may include crossbar switches,controllers, and other network nodes or components that providehigh-speed, low-latency packet routing. As depicted, host machine 152 isa source node for data transmission, whereas host machine 156 is adestination node.

In an embodiment, source-node host machine 152 may select from amongpaths 170 and 172 for each packet to be transmitted. In anotherembodiment, host machine 152 may employ a dispersive routing techniquein which multiple addresses are assigned to destination-node hostmachine 156, and where the fabrics 160-166 may associate each of thoseaddresses with different routes. Accordingly, source-node host machine152 may utilize one or more techniques for varying the path that a givenpacket to be transmitted is to take. Notably, source-node host machine152 may lack the ability to define specific routes that a packet is tofollow within any of the fabrics, relying instead on defined routesassociated with the destination address. These defined routes may bestatic or dynamic, depending on the implementation of the fabric(s).

FIG. 2 is a block diagram illustrating a host machine in the exampleform of a general-purpose computer system 200. In certain embodiments,programming of the computer system 200 according to one or moreparticular algorithms produces a special-purpose machine upon executionof that programming. In a networked deployment, the host machine mayoperate in the capacity of either a server or a client machine inserver-client network environments, or it may act as a peer machine inpeer-to-peer (or distributed) network environments. The host machine maytake any suitable form factor, such as a personal computer (PC)workstation, a server, whether rack-mounted, or stand-alone, a mainframecomputer, a cluster computing system, or the like, a set-top box, aswell as a mobile or portable computing system, such as a laptop/notebookPC, an onboard vehicle system, wearable device, a tablet PC, a hybridtablet, a personal digital assistant (PDA), a mobile telephone, anInternet of Things (IoT) device or, more generally, any machine capableof executing instructions (sequential or otherwise) that specify actionsto be taken by that machine.

Example host machine 200 includes at least one processor 202 (e.g., acentral processing unit (CPU), a graphics processing unit (GPU) or both,processor cores, compute nodes, etc.), a main memory 204 and a staticmemory 206, which communicate with each other via a link 208 (e.g.,bus). The host machine 200 may further include a video display unit 210,an alphanumeric input device 212 (e.g., a keyboard), and a userinterface (UI) navigation device 214 (e.g., a mouse). In one embodiment,the video display unit 210, input device 212 and UI navigation device214 are incorporated into a touch screen display. The host machine 200may additionally include a storage device 216 (e.g., a drive unit), asignal generation device 218 (e.g., a speaker), a network interfacedevice (NID) 220, and one or more sensors (not shown), such as a globalpositioning system (GPS) sensor, compass, accelerometer, or othersensor.

The storage device 216 includes a machine-readable medium 222 on whichis stored one or more sets of data structures and instructions 224(e.g., software) embodying or utilized by any one or more of themethodologies or functions described herein. The instructions 224 mayalso reside, completely or at least partially, within the main memory204, static memory 206, and/or within the processor 202 during executionthereof by the host machine 200, with the main memory 204, static memory206, and the processor 202 also constituting machine-readable media.

While the machine-readable medium 222 is illustrated in an exampleembodiment to be a single medium, the term “machine-readable medium” mayinclude a single medium or multiple media (e.g., a centralized ordistributed database, and/or associated caches and servers) that storethe one or more instructions 224. The term “machine-readable medium”shall also be taken to include any tangible medium that is capable ofstoring, encoding or carrying instructions for execution by the machineand that cause the machine to perform any one or more of themethodologies of the present disclosure or that is capable of storing,encoding or carrying data structures utilized by or associated with suchinstructions. The term “machine-readable medium” shall accordingly betaken to include, but not be limited to, solid-state memories, andoptical and magnetic media. Specific examples of machine-readable mediainclude non-volatile memory, including but not limited to, by way ofexample, semiconductor memory devices (e.g., electrically programmableread-only memory (EPROM), electrically erasable programmable read-onlymemory (EEPROM)) and flash memory devices (e.g., solid-state drive,removable memory card, USB-drive, integrated flash device on amotherboard, etc.); magnetic disks such as internal hard disks andremovable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

NID 220 according to various embodiments may take any suitable formfactor. In one such embodiment, NID 220 is in the form of a networkinterface card (NIC) or host-fabric interface (HFI) that interfaces withprocessor 202 via link 208. In one example, link 208 includes a PCIExpress (PCIe) bus, including a slot into which the NIC form-factor mayremovably engage. In another embodiment, NID 220 is a network interfacecircuit laid out on a motherboard together with local link circuitry,processor interface circuitry, other input/output circuitry, memorycircuitry, storage device and peripheral controller circuitry, and thelike. In another embodiment, NID 220 is a peripheral that interfaceswith link 208 via a peripheral input/output port such as a universalserial bus (USB) port. In various embodiments, a system may have oneNID, or two or more NIDs to provide multiple transmission paths to andfrom the system. In a related embodiment, one or more NIDs may beincorporated in processor 202.

One aspect of the embodiments is directed to a fast (e.g.,near-real-time) and unobtrusive (e.g., lightweight) determination of thequality of paths used for sending packets from source to destination. Inan embodiment, the destination, and intermediary devices situated alonga path taken by a packet cooperate to inform the source device about thepath quality. In a related embodiment, the packet itself it used tocarry forward to the destination information reporting any errorsexperienced along its path. The automatic repeat request (ARQ)-relatedmessaging, such as e.g., acknowledgement (ACK) and non-acknowledgement(NACK) packets, are in turn used to relay the error-reportinginformation to the source device.

In a related embodiment, the source device gathers statisticalinformation over time for the destination device (and for otherdestination devices), to determine a path quality score associated witheach path. Notably, the source device has some ability to initiate, orto perform, path selection, whether using different ports associatedwith distinct parallel paths, dispersive routing techniques, sourcerouting (e.g., full-path or partial), or other suitable technique, suchas requesting a centralized manager of a network fabric to institute arouting change in the fabric.

Advantageously, embodiments of the disclosure provide visibility as tothe path quality to applications running on the source device thatutilize the network. The path quality information is distinct fromcongestion-related state information in a number of ways. For one, alink along the path of the packets sent to the destination may beexperiencing errors, but in the absence of a large volume of traffic,the link may not be experiencing any congestion. Still, the sourcedevice benefits from knowing about the path quality because inhigh-traffic situations the congestion may be exacerbated by poor linkquality.

Also, congestion indicia does not necessarily represent a low-qualitypath; congestion may be present in situations where a particular link isexperiencing high traffic flows. Moreover, deteriorating path quality isindicative of a likelihood of link failure. Knowing the path qualitypermits the source device, and the applications running on it, toproactively change paths before any congestion- or link-failure-relatedproblems arise that may adversely impact the communication performanceover the network. This may be particularly beneficial in networks wherere-establishing a lost link is resource-intensive, which may result inunacceptable latency for performance-sensitive applications.

Additionally, the lightweight nature of the solution according to someembodiments avoids the exchange of extraneous “health packets” toprovide insight into the path quality. Furthermore, embodiments of thedisclosure may avoid having to conduct deep packet inspection, such ashaving intermediary nodes read the content of the packet payloads toobtain the path quality information. To this end, in some embodiments,one or more bits of the packet header are assigned to represent apacket-specific path-quality performance measure.

FIG. 3 is a block diagram illustrating certain components of source node300 that obtains, and responds to, path quality indicia according to anexample embodiment. Path quality indicator assessor 302 is constructed,programmed, or otherwise configured, to read incoming ACK or NACKpackets sent by the destination device to obtain a path quality measurecontained in that packet. In one example, the path quality measure maysimply be a binary indicator of an error having been detected (e.g.,exceeding a defined threshold) in the path of the outgoing packetaddressed to the destination device, the receipt of which is respondedto by the ACK or NACK packet. In other embodiments, the path qualitymeasure may have a graded value corresponding to the quantity, degree,or severity, of error(s) encountered along the path taken by thecorresponding packet delivered to the destination, or measured by anintermediary device along the path based on error experienced in thetransmission of one or more prior packets. For instance, the pathquality indicator may be a counter that is incremented each time anerror is detected by an intermediary device along the path, that exceedsa defined threshold.

Notably, the path quality indicator may be set or incremented in a givenpacket even when that particular packet experienced no errors; in thiscase, the cause for setting or incrementing the path quality indicatormay be a known problem with link quality (which may be intermittent)—asknown by an intermediary device along the path of the packet. In arelated embodiment, the intermediary device logs the occurrences oferrors (or other monitored events impacting the path quality), withtimestamps. A backwards-facing time window may be defined, in whichthere may have been errors or other adverse events in the transmissionof prior packets. The intermediary device may set or increment the pathquality indicator in response to the adverse events. In someembodiments, a scoring threshold or other path quality-based conditionis applied by the intermediary device to determine whether a currentpacket being forwarded to its destination should be marked with apath-quality indicator.

Statistics manager 304 stores path quality assessments in path qualitydatabase 306. Path quality database 306 maintains records that associatepath quality information with an indicator of the corresponding path.For instance, the path indicator may include the destination device'saddress, along with any path-selection information used by the sourcedevice to send a packet to the destination. Also, the records mayinclude a timestamp.

Statistics manager 304 is constructed, programmed, or otherwiseconfigured, to compute aggregated statistics of the path qualitymeasures. Statistics manager 304 may compute a path quality score as afunction of time, such as a running average, for example, withrelatively greater weight given to more recent path quality measurescompared to older ones.

Path selector 308 is constructed, programmed, or otherwise configured,to respond to the path quality score by selecting a path for subsequentpackets to be sent to the same destination. The path selection is basedon selection criteria 310, which may define a path-quality thresholdbelow which the path quality is deemed sufficiently low to give rise toa call for switching paths. In a related embodiment, the path-qualitythreshold is dynamic, i.e., variable over time, such as, for example,based on a prior path quality score corresponding to one or morealternative paths that may be selected. In one example use case, thepath-quality threshold may be set to the highest path quality scoreamong the alternative paths and, if the current quality score is lessthan the prior one, path selector 308 selects the higher-scoring path.In various embodiments, path selector 308 may include componentsresiding at the application, drivers, or some combination thereof.

FIG. 4 is a block diagram illustrating certain components of anintermediary node along a path of a packet according to an exampleembodiment. The intermediary node may be a switch, router, gateway orendpoint device in a multi-hop network according to various embodiments.Error detector 402 is constructed, programmed, or otherwise configured,to sense the occurrence of an error. Generally speaking, an error may bedefined as an imperfection in the transmission of a packet over a locallink, regardless as to whether or not the packet was ultimatelysuccessfully transmitted over that link, distinct from congestion of thelink. In various embodiments, a variety of events may be defined aserrors. For example, re-transmissions of frames at the link layer, errorrecovery using forward error correction (FEC) decoding, buffering ofincomplete or un-decoded frames according hybrid ARQ (HARQ) techniques,loss of one or more lanes in parallel-lane-connectivity networks, or thelike.

Path performance measure assessor 404 is constructed, programmed, orotherwise configured, to make a quantitative assessment of the detectederror, and to compare that assessment against error threshold criteria406. The operation of path performance measure assessor 404 is to applya threshold to the severity of the error such that only significanterrors (i.e., those meeting or exceeding the threshold) are reported assuch. Errors falling below the error threshold criteria may bedisregarded according to an example.

In a related embodiment, the error threshold criteria is variable. Forinstance, it may be adjusted by a centralized manager of the networkfabric for error reporting to be more or less sensitive depending on theexpected performance of the fabric. Thus, for example, for a fabric thatis constantly operated at its designed maximum throughput and data rate,it may be acceptable to encounter errors with some regularity;accordingly, the error threshold criteria may be set high, such thatonly a severe error is to be reported. In a related embodiment, theerror threshold criteria may be adjusted dynamically based on a currentlevel of congestion such that greater congestion causes the path qualitycriteria to be made more selective (i.e., less sensitive).

Forward packet marker 408 is constructed, programmed, or otherwiseconfigured, to update the packet header to indicate the path quality, inresponse to the path performance measure assessor 404 output, prior tosending a current packet to its destination or next hop towards itsdestination. In one embodiment, forward packet marker 408 first examinesthe packet header to determine if the path quality indicator is alreadyset, or set to a particular value. Forward packet marker 408 thendetermines whether the path quality indicator is to be updated. Packetmarking criteria 410 defines the rules for setting or updating the pathquality indicator. In one example, if an assessed severity of a currenterror is less than an indicated severity of an error previously set inthe packet header, packet marking criteria 410 may call for making nochange to the preexisting value of the path quality indicator.

Notably, in various embodiments, the current packet may be marked with apath quality indicator based on a current error detected by errordetector 402 in the course of propagating the current packet towards itsdestination, or based on a previous error detected in the course ofpropagating one or more preceding packets along the same path as the onetaken by the current packet. Therefore, a current packet may be markedwith an adverse path quality indication even though the current packetdid not experience an error. Rather, in these embodiments, the link mayhave been determined to be degraded to some extent by path performancemeasure assessor 404 and, consequently, all packets passing through thelink are stochastically subject to errors. Hence, an intermediary nodethat knows itself to be degraded by virtue of having measured priorpacket errors, marks subsequent packets to indicate the degradation.Error threshold criteria 406 may include a probabilistic threshold suchas, for instance, if 20% or more of all packets experience an error, theerror determination is to be triggered.

In a related embodiment, link quality database 412 includes recordsrepresenting a history log, or statistical summary, of errors associatedwith each input port, output port, or both. The ports correspond tolinks to one or more neighboring devices that may form branches of pathsthat include the intermediary node device. Each record in link qualitydatabase 412 may also include a timestamp so that the freshness of theerror occurrences may be taken into account. For example, a timewindowing filter may be applied that excludes packet errors that areolder than a given time value from consideration for the assessment ofpath performance. In a related embodiment, a discount function isapplied to reduce a prior error's severity indication as a function ofits aging, such that more recent errors are given greater weight by pathperformance measure assessor 404.

FIG. 5 is a block diagram illustrating certain components of adestination device according to an example embodiment. Path qualityindicator assessor 502 is constructed, programmed, or otherwiseconfigured, to read the header of incoming packets to obtain the valueof the path quality indicator. The path quality indicator received inthe forward-directed packet is copied to an ACK packet by ACK packetmarker 504, based on ACK packet marking criteria 506. In a relatedembodiment, ACK packet marker 504 is further constructed, programmed, orotherwise configured, to modify the path quality indicator according toACK packet marking criteria 506. For instance, in an example embodimentwhere there are fewer bits available in the ACK packet for the pathquality indication, ACK packet marker may round the value of pathquality measure indicated in the received forward-directed packet, to berepresented by the available bits in the ACK packet. In a particularexample case where a single binary bit is available in the ACK packetfor the path quality indicator, the value of the path quality measuremay be compared against a defined threshold to achieve the roundingresult.

In various embodiments, the components described above with reference toFIGS. 3-5 are implemented as engines, circuits, components, or modules,which for the sake of consistency are termed engines, although it willbe understood that these terms may be used interchangeably. Engines maybe hardware, software, or firmware communicatively coupled to one ormore processors in order to carry out the operations described herein.Engines may be hardware engines, and as such engines may be consideredtangible entities capable of performing specified operations and may beconfigured or arranged in a certain manner. In an example, circuits maybe arranged (e.g., internally or with respect to external entities suchas other circuits) in a specified manner as an engine. In an example,the whole or part of one or more hardware processors may be configuredby firmware or software (e.g., instructions, an application portion, oran application) as an engine that operates to perform specifiedoperations. In an example, the software may reside on a machine-readablemedium. In an example, the software, when executed by the underlyinghardware of the engine, causes the hardware to perform the specifiedoperations. Accordingly, the term hardware engine is understood toencompass a tangible entity, be that an entity that is physicallyconstructed, specifically configured (e.g., hardwired), or temporarily(e.g., transitorily) configured (e.g., programmed) to operate in aspecified manner or to perform part or all of any operation describedherein.

Considering examples in which engines are temporarily configured, eachof the engines need not be instantiated at any one moment in time. Forexample, where the engines comprise a general-purpose hardware processorcore configured using software; the general-purpose hardware processorcore may be configured as respective different engines at differenttimes. Software may accordingly configure a hardware processor core, forexample, to constitute a particular engine at one instance of time andto constitute a different engine at a different instance of time.

FIG. 6 is an information flow diagram illustrating a method of operationof source device 602, destination device 608, and intermediate devicesidentified as Node A 604 and Node B 608 that constitute at least aportion of a path between source device 602 and destination device 608,according to an example embodiment. Source device 602 generates aforward-directed packet 612 addressed to destination device 608. Theforward-directed packet 612 includes a header and a body. Among otherinformation, the header includes path quality indicator 632.

At transmission 620, the forward-directed packet 612 is transmitted toNode A 604. At this stage, path quality indicator 632 is initialized, asindicated at 632A. The initialized value may be indicative of no error(i.e., “0”), or it may be some other value representing the initializedstate (e.g., “INIT”). Upon reception of transmission 620, Node A 604checks for any errors. These may be identified, for example, by recoveryof errors in the data using FEC, detection of errors in the data usinganother type of parity checks or cyclical redundancy checking (CRC),link-layer retries, loss of lane, or the like. In this example, noerrors were detected; accordingly, path quality indicator 632 may beupdated to reflect an error-free link, as indicated at 632B with a valueof “0.” Transmission 622 of forward-directed packet 612 to Node B 608contains path quality indicator value 632B.

Assuming the link between Node A 604 and Node B 606 has an error in thepresent example, Node B 606, detects the error and, in some embodiments,assesses the path performance by determining a severity level of theerror. In one example, the severity level may be based on a number ofretries, on a number of corrected bits by operation of FEC, or othermeasurable parameter. Accordingly, path quality indicator 632 is updatedto reflect the error, as indicated at 632C with a value of “1.”

The updated forward-directed packet is then transmitted to destinationdevice 608 via transmission 624. Destination device 608 reads the pathquality indicator 632C, and optionally assesses its severity or impacton its operations. Also, in some embodiments, destination device 608also performs error detection and assessment for the link between Node B606 and destination device 608, and may further revise the path qualityindicator based on the result of this assessment.

Destination device 608 forms ACK packet 614, containing path qualityindicator 634. As indicated at 640, destination device 608 bases thevalue of path quality indicator 634 on received path quality indicatorvalue 632C. In one such embodiment, path quality indicator value 632C issimply copied over to path quality indicator 634 of ACK packet 614. ACKpacket 614 is addressed to source device 602, and is routed via Node B606 and Node A 604 to source device 602 via transmissions 626, 628, and630 as illustrated. Path quality indicator 634 is not updated in the ACKpacket propagation path. Accordingly, for each hop, path qualityindicators 634A, 634B, and 634C are not updated. Rather, their value issimply carried to source device 62 with each hop.

Upon receipt of ACK packet 614, source device 602 assesses the pathquality, logs the path quality indicator value, computes statisticsbased on multiple path quality indicators from multiple packetsassociated with the path, and selects a path for future transmissions todestination device 608, consistent with the embodiments discussed above,for example.

In a related embodiment, source device 602 does not perform pathselection after receiving an ACK for each and every transmitted packet.Rather, source device 602 may periodically, or in response to atriggering event, perform a path re-assessment in which the statisticsrepresenting path quality over a large number of packets, or over somesignificant period of time, are computed. This practical approach tendsto limit the overhead associated with establishing new links along anewly-selected path.

In another related embodiment, source device 602 responds to pathquality assessment by interacting with a centralized manager device,such as a centralized manager of a fabric, for instance. As an exampleoperation, an assessment of path quality degradation by source device602 may give rise to more in-depth network performance analysis by thecentralized manager, which may result in an update to the routingtables, or in the institution, or rescission, of adaptive routing, forexample. Likewise, the central manager may adjust the error-detection orpath-quality reporting criteria in response to a re-evaluation of theperformance of a particular cluster of the network.

In another type of network embodiment, a peer-peer gossip protocol fordisseminating health-related information may be utilized. For instance,each intermediary device may use a broadcast communication, or floodingcommunication mechanism, to share its knowledge regarding the pathquality to other nodes in the network. This type of embodiment may beutilized in a decentralized, distributed-managed network.

ADDITIONAL NOTES & EXAMPLES

Example 1 is a network node device apparatus, the apparatus comprising:processing hardware interfaced with a network interface device, theprocessing hardware to: originate a packet to be forward-propagatedalong a path to a destination device, the path including one or moreintermediary node devices; receive, via the network interface device, apath quality indication in an acknowledgement packet, the path qualityindication representing a reported error having been detected by atleast one of the intermediary node devices residing along the path, thereported error having been incorporated in the forward-propagatingpacket by the at least one intermediary device, received by thedestination device, and incorporated in the acknowledgement packetdirected to the network node device; and in response to the path qualityindication, initiate a path selection operation to adjust the path to betaken by future packets sourced by the network node device and directedto the destination device.

In Example 2, the subject matter of Example 1 optionally includeswherein the path quality indication represents a reported error havingbeen previously detected by the at least one of the intermediary nodedevices in propagating one or more preceding packets along the path.

In Example 3, the subject matter of any one or more of Examples 1-2optionally include wherein the path quality indication represents areported error having been detected by the at least one of theintermediary node devices in propagating the packet along the path.

In Example 4, the subject matter of any one or more of Examples 1-3optionally include wherein the path quality indication is in a header ofthe forward-propagating packet.

In Example 5, the subject matter of any one or more of Examples 1-4optionally include wherein the path quality indication is in a header ofthe acknowledgement packet.

In Example 6, the subject matter of any one or more of Examples 1-5optionally include wherein the path quality indication is a single-bitbinary value.

In Example 7, the subject matter of any one or more of Examples 1-6optionally include wherein the path quality indication is a graded valuecorresponding to a severity of the reported error.

In Example 8, the subject matter of any one or more of Examples 1-7optionally include wherein the path quality indication is a graded valuecorresponding to a probability of an error.

In Example 9, the subject matter of any one or more of Examples 1-8optionally include a statistics manager to compute aggregated statisticsof a plurality of path quality measures corresponding to respectivepaths to the destination device.

In Example 10, the subject matter of any one or more of Examples 1-9optionally include a path quality database to maintain records thatassociate path quality information with corresponding paths to aplurality of destination devices.

In Example 11, the subject matter of any one or more of Examples 1-10optionally include a path selector to initiate the path selection;wherein the path selection is made from among a plurality of networkfabrics containing paths to the destination device.

In Example 12, the subject matter of any one or more of Examples 1-11optionally include a path selector to initiate the path selection;wherein the path selection is made using dispersive routing wherein thedestination device is addressable using multiple different addresses,each address corresponding to a different path.

In Example 13, the subject matter of any one or more of Examples 1-12optionally include wherein the reported error represents one or morelink-layer retries.

In Example 14, the subject matter of any one or more of Examples 1-13optionally include wherein the reported error represents recovery ofdata using forward error-correction decoding.

In Example 15, the subject matter of any one or more of Examples 1-14optionally include wherein the path selection operation is performed bya centralized manager of a network fabric cluster.

Example 16 is a network node device apparatus, the apparatus comprising:processing hardware interfaced with a network interface device, theprocessing hardware to: receive a forward-propagating packet originatedfrom a source device, the forward-propagating packet to be furtherpropagated along a path to a destination device; assess a measure ofpath quality based on at least one detected error in communication ofone or more packets along the path; in response to assessment of themeasure of path quality, mark the forward-propagating packet with a pathquality indication based on the measure of path quality, wherein thepath quality indication is to be received by the destination device, andincorporated in an acknowledgement packet directed to the source device.

In Example 17, the subject matter of Example 16 optionally includeswherein the path quality indication represents a reported error havingbeen previously detected by the apparatus in propagating one or morepreceding packets along the path.

In Example 18, the subject matter of any one or more of Examples 16-17optionally include wherein the path quality indication represents areported error having been detected by the apparatus in propagating theforward-propagating packet along the path.

In Example 19, the subject matter of any one or more of Examples 16-18optionally include wherein the path quality indication is in a header ofthe forward-propagating packet.

In Example 20, the subject matter of any one or more of Examples 16-19optionally include wherein the path quality indication is in a header ofthe acknowledgement packet.

In Example 21, the subject matter of any one or more of Examples 16-20optionally include wherein the path quality indication is a single-bitbinary value.

In Example 22, the subject matter of any one or more of Examples 16-21optionally include wherein the path quality indication is a graded valuecorresponding to a severity of the at least one detected error.

In Example 23, the subject matter of any one or more of Examples 16-22optionally include wherein the path quality indication is a graded valuecorresponding to a probability of an error.

In Example 24, the subject matter of any one or more of Examples 16-23optionally include a link quality database to maintain records thatassociate link quality information with corresponding links toneighboring devices.

In Example 25, the subject matter of any one or more of Examples 16-24optionally include wherein the error represents one or more link-layerretries.

In Example 26, the subject matter of any one or more of Examples 16-25optionally include wherein the error represents recovery of data usingforward error-correction decoding.

Example 27 is a network node device apparatus, the apparatus comprising:processing hardware interfaced with a network interface device, theprocessing hardware to: receive a packet having been originated by asource device and forward-propagated along a path from the source deviceto the apparatus, the path including one or more intermediary nodedevices; the packet including a path quality indication representing areported error having been detected by at least one of the intermediarynode devices residing along the path, the reported error having beenincorporated in the forward-propagating packet by the at least oneintermediary device, and in response to receipt of the packet, generatean acknowledgement packet directed to the source device, theacknowledgement packet including the path quality indication.

In Example 28, the subject matter of Example 27 optionally includeswherein the path quality indication represents a reported error havingbeen previously detected by the at least one of the intermediary nodedevices in propagating one or more preceding packets along the path.

In Example 29, the subject matter of any one or more of Examples 27-28optionally include wherein the path quality indication represents areported error having been detected by the at least one of theintermediary node devices in propagating the packet along the path.

In Example 30, the subject matter of any one or more of Examples 27-29optionally include wherein the path quality indication is in a header ofthe forward-propagating packet.

In Example 31, the subject matter of any one or more of Examples 27-30optionally include wherein the path quality indication is in a header ofthe acknowledgement packet.

In Example 32, the subject matter of any one or more of Examples 27-31optionally include wherein the path quality indication is a single-bitbinary value.

In Example 33, the subject matter of any one or more of Examples 27-32optionally include wherein the path quality indication is a graded valuecorresponding to a severity of the reported error.

In Example 34, the subject matter of any one or more of Examples 27-33optionally include wherein the path quality indication is a graded valuecorresponding to a probability of an error.

In Example 35, the subject matter of any one or more of Examples 27-34optionally include wherein the reported error represents one or morelink-layer retries.

In Example 36, the subject matter of any one or more of Examples 27-35optionally include wherein the reported error represents recovery ofdata using forward error-correction decoding.

Example 37 is a machine-implemented method for communicating packets ina network, the method comprising: originating, by a source device, apacket to be forward-propagated along a path to a destination device,the path including one or more intermediary node devices; receiving, bythe source device, a path quality indication in an acknowledgementpacket, the path quality indication representing a reported error havingbeen detected by at least one of the intermediary node devices residingalong the path, the reported error having been incorporated in theforward-propagating packet by the at least one intermediary device,received by the destination device, and incorporated in theacknowledgement packet directed to the source device; and in response tothe path quality indication, initiating a path selection operation toadjust the path to be taken by future-sourced packets directed to thedestination device.

In Example 38, the subject matter of Example 37 optionally includeswherein the path quality indication represents a reported error havingbeen previously detected by the at least one of the intermediary nodedevices in propagating one or more preceding packets along the path.

In Example 39, the subject matter of any one or more of Examples 37-38optionally include wherein the path quality indication represents areported error having been detected by the at least one of theintermediary node devices in propagating the packet along the path.

In Example 40, the subject matter of any one or more of Examples 37-39optionally include wherein the path quality indication is in a header ofthe forward-propagating packet.

In Example 41, the subject matter of any one or more of Examples 37-40optionally include wherein the path quality indication is in a header ofthe acknowledgement packet.

In Example 42, the subject matter of any one or more of Examples 37-41optionally include wherein the path quality indication is a single-bitbinary value.

In Example 43, the subject matter of any one or more of Examples 37-42optionally include wherein the path quality indication is a graded valuecorresponding to a severity of the reported error.

In Example 44, the subject matter of any one or more of Examples 37-43optionally include wherein the path quality indication is a graded valuecorresponding to a probability of an error.

In Example 45, the subject matter of any one or more of Examples 37-44optionally include computing aggregated statistics of a plurality ofpath quality measures corresponding to respective paths to thedestination device.

In Example 46, the subject matter of any one or more of Examples 37-45optionally include maintaining records that associate path qualityinformation with corresponding paths to a plurality of destinationdevices.

In Example 47, the subject matter of any one or more of Examples 37-46optionally include wherein the path selection is made from among aplurality of network fabrics containing paths to the destination device.

In Example 48, the subject matter of any one or more of Examples 37-47optionally include wherein the path selection is made using dispersiverouting wherein the destination device is addressable using multipledifferent addresses, each address corresponding to a different path.

In Example 49, the subject matter of any one or more of Examples 37-48optionally include wherein the reported error represents one or morelink-layer retries.

In Example 50, the subject matter of any one or more of Examples 37-49optionally include wherein the reported error represents recovery ofdata using forward error-correction decoding.

In Example 51, the subject matter of any one or more of Examples 37-50optionally include wherein the path selection operation is performed bya centralized manager of a network fabric cluster.

In Example 52, the subject matter of any one or more of Examples 37-51optionally include wherein the originating, receiving and initiating areall performed autonomously by the same machine, the machine being anetwork node device.

Example 53 is a machine-implemented method for communicating packets ina network, the method comprising: receiving a forward-propagating packetoriginated from a source device, the forward-propagating packet to befurther propagated along a path to a destination device; assessing ameasure of path quality based on at least one detected error incommunication of one or more packets along the path; and in response toassessment of the measure of path quality, marking theforward-propagating packet with a path quality indication based on themeasure of path quality, wherein the path quality indication is to bereceived by the destination device, and incorporated in anacknowledgement packet directed to the source device; wherein thereceiving, assessing, and marking are each performed autonomously by amachine.

In Example 54, the subject matter of Example 53 optionally includeswherein the path quality indication represents a reported error havingbeen previously detected by the method in propagating one or morepreceding packets along the path.

In Example 55, the subject matter of any one or more of Examples 53-54optionally include wherein the path quality indication represents areported error having been detected by the method in propagating theforward-propagating packet along the path.

In Example 56, the subject matter of any one or more of Examples 53-55optionally include wherein the path quality indication is in a header ofthe forward-propagating packet.

In Example 57, the subject matter of any one or more of Examples 53-56optionally include wherein the path quality indication is in a header ofthe acknowledgement packet.

In Example 58, the subject matter of any one or more of Examples 53-57optionally include wherein the path quality indication is a single-bitbinary value.

In Example 59, the subject matter of any one or more of Examples 53-58optionally include wherein the path quality indication is a graded valuecorresponding to a severity of the at least one detected error.

In Example 60, the subject matter of any one or more of Examples 53-59optionally include wherein the path quality indication is a graded valuecorresponding to a probability of an error.

In Example 61, the subject matter of any one or more of Examples 53-60optionally include maintaining records that associate link qualityinformation with corresponding links to neighboring devices.

In Example 62, the subject matter of any one or more of Examples 53-61optionally include wherein the error represents one or more link-layerretries.

In Example 63, the subject matter of any one or more of Examples 53-62optionally include wherein the error represents recovery of data usingforward error-correction decoding.

Example 64 is a machine-implemented method for communicating packets ina network, the method comprising: receiving, by a destination nodedevice, a packet having been originated by a source device andforward-propagated along a path from the source device to thedestination node device, the path including one or more intermediarynode devices; the packet including a path quality indicationrepresenting a reported error having been detected by at least one ofthe intermediary node devices residing along the path, the reportederror having been incorporated in the forward-propagating packet by theat least one intermediary device, and in response to receipt of thepacket, generating, by the destination node device, an acknowledgementpacket directed to the source device, the acknowledgement packetincluding the path quality indication.

In Example 65, the subject matter of Example 64 optionally includeswherein the path quality indication represents a reported error havingbeen previously detected by the at least one of the intermediary nodedevices in propagating one or more preceding packets along the path.

In Example 66, the subject matter of any one or more of Examples 64-65optionally include wherein the path quality indication represents areported error having been detected by the at least one of theintermediary node devices in propagating the packet along the path.

In Example 67, the subject matter of any one or more of Examples 64-66optionally include wherein the path quality indication is in a header ofthe forward-propagating packet.

In Example 68, the subject matter of any one or more of Examples 64-67optionally include wherein the path quality indication is in a header ofthe acknowledgement packet.

In Example 69, the subject matter of any one or more of Examples 64-68optionally include wherein the path quality indication is a single-bitbinary value.

In Example 70, the subject matter of any one or more of Examples 64-69optionally include wherein the path quality indication is a graded valuecorresponding to a severity of the reported error.

In Example 71, the subject matter of any one or more of Examples 64-70optionally include wherein the path quality indication is a graded valuecorresponding to a probability of an error.

In Example 72, the subject matter of any one or more of Examples 64-71optionally include wherein the reported error represents one or morelink-layer retries.

In Example 73, the subject matter of any one or more of Examples 64-72optionally include wherein the reported error represents recovery ofdata using forward error-correction decoding.

Example 74 is at least one machine-readable medium comprising:instructions that, when executed by at least one machine havingcomputing and network-interface hardware, cause the at least one machineto: originate a packet to be forward-propagated along a path to adestination device, the path including one or more intermediary nodedevices; receive a path quality indication in an acknowledgement packet,the path quality indication representing a reported error having beendetected by at least one of the intermediary node devices residing alongthe path, the reported error having been incorporated in theforward-propagating packet by the at least one intermediary device,received by the destination device, and incorporated in theacknowledgement packet; and in response to the path quality indication,initiate a path selection operation to adjust the path to be taken byfuture packets sourced by the network node device and directed to thedestination device.

In Example 75, the subject matter of Example 74 optionally includeswherein the path quality indication represents a reported error havingbeen previously detected by the at least one of the intermediary nodedevices in propagating one or more preceding packets along the path.

In Example 76, the subject matter of any one or more of Examples 74-75optionally include wherein the path quality indication represents areported error having been detected by the at least one of theintermediary node devices in propagating the packet along the path.

In Example 77, the subject matter of any one or more of Examples 74-76optionally include wherein the path quality indication is in a header ofthe forward-propagating packet.

In Example 78, the subject matter of any one or more of Examples 74-77optionally include wherein the path quality indication is in a header ofthe acknowledgement packet.

In Example 79, the subject matter of any one or more of Examples 74-78optionally include wherein the path quality indication is a single-bitbinary value.

In Example 80, the subject matter of any one or more of Examples 74-79optionally include wherein the path quality indication is a graded valuecorresponding to a severity of the reported error.

In Example 81, the subject matter of any one or more of Examples 74-80optionally include wherein the path quality indication is a graded valuecorresponding to a probability of an error.

In Example 82, the subject matter of any one or more of Examples 74-81optionally include instructions for computing aggregated statistics of aplurality of path quality measures corresponding to respective paths tothe destination device.

In Example 83, the subject matter of any one or more of Examples 74-82optionally include instructions for maintaining records that associatepath quality information with corresponding paths to a plurality ofdestination devices.

In Example 84, the subject matter of any one or more of Examples 74-83optionally include wherein the path selection is made from among aplurality of network fabrics containing paths to the destination device.

In Example 85, the subject matter of any one or more of Examples 74-84optionally include wherein the path selection is made using dispersiverouting wherein the destination device is addressable using multipledifferent addresses, each address corresponding to a different path.

In Example 86, the subject matter of any one or more of Examples 74-85optionally include wherein the reported error represents one or morelink-layer retries.

In Example 87, the subject matter of any one or more of Examples 74-86optionally include wherein the reported error represents recovery ofdata using forward error-correction decoding.

In Example 88, the subject matter of any one or more of Examples 74-87optionally include wherein the path selection operation is performed bya centralized manager of a network fabric cluster.

In Example 89, the subject matter of any one or more of Examples 74-88optionally include wherein the originating, receiving and initiating areall performed autonomously by the same machine, the machine being anetwork node device.

Example 90 is at least one machine-readable medium, comprising:instructions that, when executed by at least one machine havingcomputing and network-interface hardware, cause the at least one machineto: receive a forward-propagating packet originated from a sourcedevice, the forward-propagating packet to be further propagated along apath to a destination device; assess a measure of path quality based onat least one detected error in communication of one or more packetsalong the path; and in response to assessment of the measure of pathquality, mark the forward-propagating packet with a path qualityindication based on the measure of path quality, wherein the pathquality indication is to be received by the destination device, andincorporated in an acknowledgement packet directed to the source device.

In Example 91, the subject matter of Example 90 optionally includeswherein the path quality indication represents a reported error havingbeen previously detected by the at least one machine-readable medium inpropagating one or more preceding packets along the path.

In Example 92, the subject matter of any one or more of Examples 90-91optionally include wherein the path quality indication represents areported error having been detected by the at least one machine-readablemedium in propagating the forward-propagating packet along the path.

In Example 93, the subject matter of any one or more of Examples 90-92optionally include wherein the path quality indication is in a header ofthe forward-propagating packet.

In Example 94, the subject matter of any one or more of Examples 90-93optionally include wherein the path quality indication is in a header ofthe acknowledgement packet.

In Example 95, the subject matter of any one or more of Examples 90-94optionally include wherein the path quality indication is a single-bitbinary value.

In Example 96, the subject matter of any one or more of Examples 90-95optionally include wherein the path quality indication is a graded valuecorresponding to a severity of the at least one detected error.

In Example 97, the subject matter of any one or more of Examples 90-96optionally include wherein the path quality indication is a graded valuecorresponding to a probability of an error.

In Example 98, the subject matter of any one or more of Examples 90-97optionally include instructions for maintaining records that associatelink quality information with corresponding links to neighboringdevices.

In Example 99, the subject matter of any one or more of Examples 90-98optionally include wherein the error represents one or more link-layerretries.

In Example 100, the subject matter of any one or more of Examples 90-99optionally include wherein the error represents recovery of data usingforward error-correction decoding.

Example 101 is at least one machine-readable medium, comprising:instructions that, when executed by a machine having computing andnetwork-interface hardware, cause the machine to: receive a packethaving been originated by a source device and forward-propagated along apath from the source device to the machine, the path including one ormore intermediary node devices; the packet including a path qualityindication representing a reported error having been detected by atleast one of the intermediary node devices residing along the path, thereported error having been incorporated in the forward-propagatingpacket by the at least one intermediary device, and in response toreceipt of the packet, generate an acknowledgement packet directed tothe source device, the acknowledgement packet including the path qualityindication.

In Example 102, the subject matter of Example 101 optionally includeswherein the path quality indication represents a reported error havingbeen previously detected by the at least one of the intermediary nodedevices in propagating one or more preceding packets along the path.

In Example 103, the subject matter of any one or more of Examples101-102 optionally include wherein the path quality indicationrepresents a reported error having been detected by the at least one ofthe intermediary node devices in propagating the packet along the path.

In Example 104, the subject matter of any one or more of Examples101-103 optionally include wherein the path quality indication is in aheader of the forward-propagating packet.

In Example 105, the subject matter of any one or more of Examples101-104 optionally include wherein the path quality indication is in aheader of the acknowledgement packet.

In Example 106, the subject matter of any one or more of Examples101-105 optionally include wherein the path quality indication is asingle-bit binary value.

In Example 107, the subject matter of any one or more of Examples101-106 optionally include wherein the path quality indication is agraded value corresponding to a severity of the reported error.

In Example 108, the subject matter of any one or more of Examples101-107 optionally include wherein the path quality indication is agraded value corresponding to a probability of an error.

In Example 109, the subject matter of any one or more of Examples101-108 optionally include wherein the reported error represents one ormore link-layer retries.

In Example 110, the subject matter of any one or more of Examples101-109 optionally include wherein the reported error representsrecovery of data using forward error-correction decoding.

Example 111 is a system for communicating packets in a network, thesystem comprising: means for originating, by a source device, a packetto be forward-propagated along a path to a destination device, the pathincluding one or more intermediary node devices; means for receiving, bythe source device, a path quality indication in an acknowledgementpacket, the path quality indication representing a reported error havingbeen detected by at least one of the intermediary node devices residingalong the path, the reported error having been incorporated in theforward-propagating packet by the at least one intermediary device,received by the destination device, and incorporated in theacknowledgement packet directed to the source device; and means forinitiating a path selection operation, in response to the path qualityindication, the path selection operation to adjust the path to be takenby future-sourced packets directed to the destination device.

In Example 112, the subject matter of Example 111 optionally includeswherein the path quality indication represents a reported error havingbeen previously detected by the at least one of the intermediary nodedevices in propagating one or more preceding packets along the path.

In Example 113, the subject matter of any one or more of Examples111-112 optionally include wherein the path quality indicationrepresents a reported error having been detected by the at least one ofthe intermediary node devices in propagating the packet along the path.

In Example 114, the subject matter of any one or more of Examples111-113 optionally include wherein the path quality indication is in aheader of the forward-propagating packet.

In Example 115, the subject matter of any one or more of Examples111-114 optionally include wherein the path quality indication is in aheader of the acknowledgement packet.

In Example 116, the subject matter of any one or more of Examples111-115 optionally include wherein the path quality indication is asingle-bit binary value.

In Example 117, the subject matter of any one or more of Examples111-116 optionally include wherein the path quality indication is agraded value corresponding to a severity of the reported error.

In Example 118, the subject matter of any one or more of Examples111-117 optionally include wherein the path quality indication is agraded value corresponding to a probability of an error.

In Example 119, the subject matter of any one or more of Examples111-118 optionally include means for computing aggregated statistics ofa plurality of path quality measures corresponding to respective pathsto the destination device.

In Example 120, the subject matter of any one or more of Examples111-119 optionally include means for maintaining records that associatepath quality information with corresponding paths to a plurality ofdestination devices.

In Example 121, the subject matter of any one or more of Examples111-120 optionally include wherein the path selection is made from amonga plurality of network fabrics containing paths to the destinationdevice.

In Example 122, the subject matter of any one or more of Examples111-121 optionally include wherein the path selection is made usingdispersive routing wherein the destination device is addressable usingmultiple different addresses, each address corresponding to a differentpath.

In Example 123, the subject matter of any one or more of Examples111-122 optionally include wherein the reported error represents one ormore link-layer retries.

In Example 124, the subject matter of any one or more of Examples111-123 optionally include wherein the reported error representsrecovery of data using forward error-correction decoding.

In Example 125, the subject matter of any one or more of Examples111-124 optionally include wherein the path selection operation isperformed by a centralized manager of a network fabric cluster.

In Example 126, the subject matter of any one or more of Examples111-125 optionally include wherein the originating, receiving andinitiating are all performed autonomously by the same machine, themachine being a network node device.

Example 127 is a system for communicating packets in a network, thesystem comprising: means for receiving a forward-propagating packetoriginated from a source device, the forward-propagating packet to befurther propagated along a path to a destination device; means forassessing a measure of path quality based on at least one detected errorin communication of one or more packets along the path; and means formarking the forward-propagating packet with a path quality indication inresponse to assessment of the measure of path quality, the path qualityindication being based on the measure of path quality, wherein the pathquality indication is to be received by the destination device, andincorporated in an acknowledgement packet directed to the source device.

In Example 128, the subject matter of Example 127 optionally includeswherein the path quality indication represents a reported error havingbeen previously detected by the system in propagating one or morepreceding packets along the path.

In Example 129, the subject matter of any one or more of Examples127-128 optionally include wherein the path quality indicationrepresents a reported error having been detected by the system inpropagating the forward-propagating packet along the path.

In Example 130, the subject matter of any one or more of Examples127-129 optionally include wherein the path quality indication is in aheader of the forward-propagating packet.

In Example 131, the subject matter of any one or more of Examples127-130 optionally include wherein the path quality indication is in aheader of the acknowledgement packet.

In Example 132, the subject matter of any one or more of Examples127-131 optionally include wherein the path quality indication is asingle-bit binary value.

In Example 133, the subject matter of any one or more of Examples127-132 optionally include wherein the path quality indication is agraded value corresponding to a severity of the at least one detectederror.

In Example 134, the subject matter of any one or more of Examples127-133 optionally include wherein the path quality indication is agraded value corresponding to a probability of an error.

In Example 135, the subject matter of any one or more of Examples127-134 optionally include means for maintaining records that associatelink quality information with corresponding links to neighboringdevices.

In Example 136, the subject matter of any one or more of Examples127-135 optionally include wherein the error represents one or morelink-layer retries.

In Example 137, the subject matter of any one or more of Examples127-136 optionally include wherein the error represents recovery of datausing forward error-correction decoding.

Example 138 is a system for communicating packets in a network, thesystem comprising: means for receiving, by a destination node device, apacket having been originated by a source device and forward-propagatedalong a path from the source device to the destination node device, thepath including one or more intermediary node devices; the packetincluding a path quality indication representing a reported error havingbeen detected by at least one of the intermediary node devices residingalong the path, the reported error having been incorporated in theforward-propagating packet by the at least one intermediary device, andmeans for generating an acknowledgement packet directed to the sourcedevice in response to receipt of the packet, the acknowledgement packetincluding the path quality indication.

In Example 139, the subject matter of Example 138 optionally includeswherein the path quality indication represents a reported error havingbeen previously detected by the at least one of the intermediary nodedevices in propagating one or more preceding packets along the path.

In Example 140, the subject matter of any one or more of Examples138-139 optionally include wherein the path quality indicationrepresents a reported error having been detected by the at least one ofthe intermediary node devices in propagating the packet along the path.

In Example 141, the subject matter of any one or more of Examples138-140 optionally include wherein the path quality indication is in aheader of the forward-propagating packet.

In Example 142, the subject matter of any one or more of Examples138-141 optionally include wherein the path quality indication is in aheader of the acknowledgement packet.

In Example 143, the subject matter of any one or more of Examples138-142 optionally include wherein the path quality indication is asingle-bit binary value.

In Example 144, the subject matter of any one or more of Examples138-143 optionally include wherein the path quality indication is agraded value corresponding to a severity of the reported error.

In Example 145, the subject matter of any one or more of Examples138-144 optionally include wherein the path quality indication is agraded value corresponding to a probability of an error.

In Example 146, the subject matter of any one or more of Examples138-145 optionally include wherein the reported error represents one ormore link-layer retries.

In Example 147, the subject matter of any one or more of Examples138-146 optionally include wherein the reported error representsrecovery of data using forward error-correction decoding.

The above detailed description includes references to the accompanyingdrawings, which form a part of the detailed description. The drawingsshow, by way of illustration, specific embodiments that may bepracticed. These embodiments are also referred to herein as “examples.”Such examples may include elements in addition to those shown ordescribed. However, also contemplated are examples that include theelements shown or described. Moreover, also contemplate are examplesusing any combination or permutation of those elements shown ordescribed (or one or more aspects thereof), either with respect to aparticular example (or one or more aspects thereof), or with respect toother examples (or one or more aspects thereof) shown or describedherein.

Publications, patents, and patent documents referred to in this documentare incorporated by reference herein in their entirety, as thoughindividually incorporated by reference. In the event of inconsistentusages between this document and those documents so incorporated byreference, the usage in the incorporated reference(s) are supplementaryto that of this document; for irreconcilable inconsistencies, the usagein this document controls.

In this document, the terms “a” or “an” are used, as is common in patentdocuments, to include one or more than one, independent of any otherinstances or usages of “at least one” or “one or more.” In thisdocument, the term “or” is used to refer to a nonexclusive or, such that“A or B” includes “A but not B,” “B but not A,” and “A and B,” unlessotherwise indicated. In the appended claims, the terms “including” and“in which” are used as the plain-English equivalents of the respectiveterms “comprising” and “wherein.” Also, in the following claims, theterms “including” and “comprising” are open-ended, that is, a system,device, article, or process that includes elements in addition to thoselisted after such a term in a claim are still deemed to fall within thescope of that claim. Moreover, in the following claims, the terms“first,” “second,” and “third,” etc. are used merely as labels, and arenot intended to suggest a numerical order for their objects.

The above description is intended to be illustrative, and notrestrictive. For example, the above-described examples (or one or moreaspects thereof) may be used in combination with others. Otherembodiments may be used, such as by one of ordinary skill in the artupon reviewing the above description. The Abstract is to allow thereader to quickly ascertain the nature of the technical disclosure andis submitted with the understanding that it will not be used tointerpret or limit the scope or meaning of the claims. Also, in theabove Detailed Description, various features may be grouped together tostreamline the disclosure. However, the claims may not set forthfeatures disclosed herein because embodiments may include a subset ofsaid features. Further, embodiments may include fewer features thanthose disclosed in a particular example. Thus, the following claims arehereby incorporated into the Detailed Description, with a claim standingon its own as a separate embodiment. The scope of the embodimentsdisclosed herein is to be determined with reference to the appendedclaims, along with the full scope of equivalents to which such claimsare entitled.

What is claimed is:
 1. A network node device apparatus, the apparatuscomprising: processing hardware interfaced with a network interfacedevice, the processing hardware to: receive a forward-propagating packetoriginated from a source device, the forward-propagating packet to befurther propagated along a path to a destination device, theforward-propagating packet containing data for the destination devicerather than being a health packet; assess a measure of path qualitybased on at least one detected error in communication of one or morepackets along the path; in response to assessment of the measure of pathquality, mark the forward-propagating packet with a path qualityindication based on the measure of path quality; and transmit theforward-propagating packet modified by the path quality indication alongthe path to the destination device, wherein the path quality indicationis to be received by the destination device and incorporated in anacknowledgement packet directed to the source device, and whereinsubsequent use of the network node device as an intermediary between thesource and destination devices is dependent on the path qualityindication in the acknowledgment packet.
 2. The apparatus of claim 1,wherein the path quality indication represents a reported error havingbeen previously detected by the apparatus in propagating one or morepreceding data packets along the path independent of whether theforward-propagating packet experiences an error.
 3. The apparatus ofclaim 1, wherein the path quality indication represents a reported errorhaving been detected by the apparatus in propagating theforward-propagating packet along the path.
 4. The apparatus of claim 1,wherein the path quality indication is in a header of theforward-propagating packet.
 5. The apparatus of claim 1, wherein thepath quality indication is in a header of the acknowledgement packet. 6.The apparatus of claim 1, wherein the path quality indication is asingle-bit binary value.
 7. The apparatus of claim 1, wherein the pathquality indication is a graded value corresponding to a severity of theat least one detected error.
 8. The apparatus of claim 1, wherein thepath quality indication is a graded value corresponding to a probabilityof an error.
 9. The apparatus of claim 1, further comprising: a linkquality database to maintain records that associate link qualityinformation with corresponding links to neighboring devices.
 10. Theapparatus of claim 1, wherein the processing hardware is furtherconfigured to: receive the acknowledgment packet from the destinationnode; avoid altering the adverse path quality indication in theacknowledgment packet independent of whether a change of the measure ofpath quality has occurred; and transmit the acknowledgment packet to thesource node.
 11. The apparatus of claim 1, wherein the processinghardware is further configured to: apply a scoring threshold todetermine whether the forward-propagating packet should be marked withthe path quality indicator.
 12. The apparatus of claim 1, wherein: thenetwork node device is a first intermediary device that initiallymodifies the path quality indication from an initial value to a valueindicated by the measure of path quality, the initial value representingan initialized state of the path quality indication.
 13. The apparatusof claim 1, wherein the processing hardware is further configured to:assess the path quality against variable error threshold criteria thatis dependent on an expected performance of a network fabric containingthe network node device.
 14. The apparatus of claim 13, wherein theprocessing hardware is further configured to at least one of: increasethe error threshold criteria if the fabric is constantly operated at adesigned maximum throughput and data rate, or dynamically adjust theerror threshold criteria based on a current level of congestion suchthat greater congestion causes the error threshold criteria to be lesssensitive.
 15. The apparatus of claim 1, wherein: the path qualityindication represents a reported error having been previously detectedby the apparatus in propagating one or more preceding data packets alongthe path, and a time windowing filter is applied that excludes packeterrors that are older than a given time value from consideration for theassessment.
 16. The apparatus of claim 15, wherein: a discount functionis applied to reduce severity of a prior error indication as a functionof a timestamp of the forward-propagating packet such that more recenterrors are given greater weight.
 17. The apparatus of claim 1, whereinthe processing hardware is further configured to use a broadcastcommunication to share information regarding the path quality to othernodes, and the broadcast communication is separate from theforward-propagating packet.
 18. At least one machine-readable medium,comprising: instructions that, when executed by at least one machinehaving computing and network-interface hardware, cause the at least onemachine to: receive a forward-propagating packet originated from asource device, the forward-propagating packet to be further propagatedalong a path to a destination device, the forward-propagating packetcontaining data for the destination device rather than being a healthpacket; assess a measure of path quality based on at least one detectederror in communication of one or more packets along the path; inresponse to assessment of the measure of path quality, mark theforward-propagating packet with a path quality indication based on themeasure of path quality; and transmit the forward-propagating packetmodified by the path quality indication along the path to thedestination device, wherein the path quality indication is to bereceived by the destination device and incorporated in anacknowledgement packet directed to the source device, and whereinsubsequent use of the network node device as an intermediary between thesource and destination devices is dependent on the path qualityindication in the acknowledgment packet.
 19. The at least onemachine-readable medium of claim 18, wherein the path quality indicationis a graded value corresponding to a probability of an error.
 20. The atleast one machine-readable medium of claim 18, wherein the errorrepresents one or more link-layer retries.